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Abstract 

We study the lattice of finite-index extensions of a given finitely gen- 
erated subgroup of a free group F. This lattice is finite and we give a 
combinatorial characterization of its greatest element, which is the com- 
mensurator of H. This characterization leads to a fast algorithm to com- 
pute the commensurator, which is based on a standard algorithm from 
automata theory. We also give a sub-exponential and super-polynomial 
upper bound for the number of finite-index extensions of H, and we give a 
language-theoretic characterization of the lattice of finite-index subgroups 
of H. Finally, we give a polynomial time algorithm to compute the mal- 
normal closure of H. 
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The particular object of study in this paper is the lattice of extensions of a 
given finitely generated subgroup H of and more specifically the sublattice of 
finite-index extensions of H. In this paper, all groups are subgroups of a fixed 
free group, and the notion of extension must be understood in this context. 

It is elementary to verify that H has only finitely many finite-index exten- 
sions, and it is known that if K and L are finite-index extensions of H , then the 
subgroup they generate, namely {K,L), has finite index over H as well (Green- 
berg's theorem, see [H]). Therefore H has a maximum finite- index extension 
Hf, , which is effectively constructible, and the finite-index extensions of H form 
a full convex sublattice of the lattice of subgroups of F. 

This paper contains a detailed discussion of the lattice of finite-index exten- 
sions of H. Our main contributions are the following. 

We show that the maximum finite-index extension Hf, of H is the commen- 
surator of H, and we give a combinatorial (graph-theoretic) characterization 
of Hf\ . This characterization leads to efficient algorithms to compute all finite- 
index extensions of H, and to compute Hf, - the latter in time 0{n\ogn). We 
also give a rather tight upper bound on the number of finite-index extensions 
of H: there are at most 0{y/n '°82 such extensions, where n is the number 
of vertices in the graphical representation of H. Note that this upper bound is 
sub-exponential but super-polynomial. 

The consideration of the subgroups of the form Hf,, which have no proper 
finite-index extensions, leads us to the dual study of the lattice of finite-index 
subgroups of a given subgroup, and we give a combinatorial (language-theoretic) 
characterization of each such lattice. 

Finally, we use our better understanding of the lattice of extensions of a 
subgroup of F, to give a polynomial time algorithm to compute the malnormal 
closure of a given subgroup. 

As we already indicated, we use in a fundamental way the graphical repre- 
sentation of finitely generated subgroups of F, including a detailed study of the 
different steps of the computation of this representation (given a set of genera- 
tors for the subgroup H), whose study was at the heart of an earlier paper by 
the authors |10j . It is particularly interesting to see that language-theoretic re- 
sults and arguments play an important role in this paper: that is, we sometimes 
consider the graphical representation of a subgroup not just as an edge-labeled 
graph, but as a finite state automaton. Such considerations are present in al- 
most all the results of this paper, but they become crucial at a rather unexpected 
juncture: the design of an efficient algorithm to compute the maximal finite- 
index extension Hf, of H. Indeed, the very low complexity we achieve is due 
to the possibility of using a standard automata-theoretic algorithm, namely the 
computation of the minimal automaton of a regular language. 

Section [T] summarizes a number of well-known facts about free groups and 
the representation of their finitely generated subgroups, which will be used freely 
in the sequel (see [TTJ [TH [SJ [71 [TD] for more details). Section [5] is the heart of the 
paper: it starts with a technical study of the different steps of the algorithm to 
compute the graphical representation of a given subgroup, and a description of 
those steps which preserve finite-index (Section |2.2|) . These technical results are 
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then used to characterize the maximal finite- index extension Hf\ fSection 12. 3p . 
to relate the computation of Hf, and the minimization of certain finite-state au- 
tomata fSection l2.4p . to evaluate the maximal number of finite-index extensions 
of a given subgroup (Section 12. 5p . and to describe an invariant of the lattice of 
finite- index subgroups of a given subgroup (Section 12. 6p . 

Finally, we apply the same ideas in Section[31 to study the malnormal closure 
of a subgroup, and to show that it can be computed in polynomial time. 

1 Subgroups of free groups and Stallings graphs 

Let F be a finitely generated free group and let A = {ai, . . . , Ur} be a fixed 
basis of F. Let A = {ai, . . . , Or} be a disjoint copy of A and let A = AU A: as 
usual, we extend the map a '—^ a from the set A to all words by letting a = a if 
a G A and ua = au ii a Q A and u G A* . As usual again, the elements of F are 
identified with the reduced words over the alphabet A, that is, the words that 
do not contain a sequence of the form aa (a G A). If u £ A* is an arbitrary 
word, we denote by red(u) the corresponding reduced word, that is, the word 
obtained from u by repeatedly deleting all sequences of the form aa (a G A). 

A reduced word u G F is cyclically reduced if u cannot be written as m = ava 
with a G A and v G F. Every reduced word u can be factored in a unique way 
in the form u = xyx, with y cyclically reduced. 

If iJ is a subgroup of F, an extension of H is any subgroup G containing 
H and we write H < G. If iJ is finitely generated, we also write H <fg G. If 
H has finite index in G, we say that G is a finite-index extension of H and we 
write H <fi G. Finally, we write H <ff G if if is a free factor of G. 

1.1 The graphical representation of a subgroup 

It is well known (since Scrrc's and Stalling's fundamental work [8l[TT]) that every 
finitely generated subgroup H <fg F admits a unique graphical representation 
of the form A{H) = {T{H), 1), where r{H) is a finite directed graph with A- 
labeled edges and 1 is a designated vertex of r(7J), subject to the combinatorial 
conditions below. Here, a graph is a pair (V, E) where V is the set of vertices 
and E <Z V x AxV is the set of edges; the in-degree (resp. out-degree) of a vertex 
V G V is the number of edges in E of the form (v',a,v) (resp. {v,a,v')); and 
the degree of v is the sum of its in- and out-degree. Every pair A{H) satisfies 
the following: 

- the (underlying undirected) graph is connected; 

- for each a G A, every vertex is the source (resp. the target) of at most one 
a-labeled edge; 

- and every vertex, except possibly 1, has degree at least 2. 

Moreover, every pair (F, 1) with these properties is said to be admissible, and it 
is the representation of a finitely generated subgroup of F. In addition, given a 
finite set of generators of H, the representation of H is effectively computable. 
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We refer the reader to [TTl [51 [71 [T^l [IHI for some of the literature on this construc- 
tion and its many applications, and to Section [2.11 below on the construction of 
A{H). 

We sometimes like to view the ^-labeled graph T{H) as a transition system 
over the alphabet A: if p, q are vertices of T{H), a G A and (p, a, q) is an edge 
of r(i?), we say that a labels a path from p to q and a labels a path from q to 
p, written p — ^ q and q p. If u G A* is a word (reduced or not) and u — va 
{a & A), we say that p — ^ q {u labels a path from p to q) if p — ^ p' q for 
some vertex p' . In particular, a reduced word is in H if and only if it labels a 
loop at vertex 1. Moreover, if we have a path p — > q, then we also have a path 

red(u) 

p > q. 

If H <fg G <fg then there is a homomorphism from A{H) into A{G), 
that is, a map (f from the vertex set of r(_ff ) to the vertex set of T{G) such that 

- (^(1) — 1 and 

- if p — ^ q in r(i/) (p, q vertices, a S A), then (p{p) ^p{q) in r(G). 

It is not difficult to verify that this morphism, if it exists, is unique, and we 
denote it by ip^. It is well known (see [H [Til El El) that if ip^ is one-to-one, 
then H <ff G. 

Finally, we say that the homomorphism is a cover, if it satisfies 

- if p, g are vertices of T{H), a £ A and ip^{p) ip'^{q) in T{G), then 
p-^q' in T{H) for some vertex such that ip'fj{q') = (p^{q). In that case, all 
sets of the form {(p'^)^^{q) {q a vertex of r(G')) have the same cardinality. 

Covers have the following property, which we will use freely in the sequel. 

Lemma 1.1 If ip: A{H) — > A{G) is a cover, p is a vertex of A{H) and u £ F 
labels a loop of A{G) at ^{p), then red(u'") labels a loop of A{H) at p for some 
integer m > 1 . 

1.2 Covers, cyclically reduced subgroups and finite-index 
extensions 

Let us say that H is cyclically reduced (with respect to the basis A) if every 
vertex of r(_ff ) has degree at least equal to 2. If H is not cyclically reduced, then 
the designated vertex 1 of T{H) has degree 1 and T{H) consists of two parts: 
tail(r(i?)), which contains the designated vertex 1 and all degree 2 vertices that 
can be connected to vertex 1 through other degree 2 vertices; and the rest of 
T{H), which is called the core of T{H), written cc{T{H)). We let ti/(l) be the 
shortest word which labels a path from 1 to a vertex in cc(r(iJ)) and let Tij(l) 
be the vertex of cc(r(_ff)) thus reached (if H is cyclically reduced, then t//(l) 
is the empty word and th{^) = 1). We write t(l) and t(1) if the subgroup H is 
clear from the context. 

The tail and the core of r(iJ) have intrinsic characterizations. The charac- 
terization of the core is well-known (see ^llj Exercise 7.3(a)]) and that of the 
tail is an elementary consequence. 
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Remark 1.2 Let H <fg F and let p be a vertex of T{H). Then p is a vertex 
of cc(r(iJ)) if and only if some cyclically reduced word u labels a path from p 
to p. □ 



Proof. By definition, if H is not cyclically reduced, then tail(r(iJ)) consists of 
a single path from vertex 1 to vertex t(l) (excluding the latter vertex): it is 
therefore elementary to verify that no non-empty cyclically reduced word labels 
a loop at a vertex in tail(r(iJ)). 

Let now p be a vertex in cc{T{H)): then p has degree at least 2, and if it 
has degree exactly 2, then neither of the two edges adjacent to it leads to a 
vertex in tail(r(H)). Therefore, one can find distinct letters a,a' Q A such that 

p-^q and p-^q', with q and q' in cc(r(iJ)) as well. Iterating this reasoning, 
one can show that there exist arbitrarily long paths within cc(r(iJ)), starting 
from p and labeled by reduced words of the form au and a'u' . Since T{H) is 
finite, vertices are repeated along these paths, and we consider the earliest such 
repetition after the initial p. If p itself is the first repeated vertex along the 
path labeled au, we have a loop p — ^p such that au is cyclically reduced, and 
we are done. The situation is similar if p is the first repeated vertex along the 
path labeled a'u' . Otherwise, let r and r' be the first repeated vertices along 
the two paths. Then r p, r' ^ p, and cc(r(iJ)) has paths of the form p — ^ r, 

p r', r — ^ r and r' r' such that auvua, a'u'v'u'a! are reduced. Then the 
word auvuaa'u'v'u'a' is cyclically reduced, and it labels a loop at p in T(H). 
This concludes the proof. □ 

Remark 1.3 Let H <fg F. Then t(l) is the maximum common prefix of the 
non-trivial elements of H. □ 

Proof. Since every non-trivial element of H is the label of a loop at 1 in T{H), 
it is clear that t(l) is a common prefix to all these words. 

By Remark 1 1.21 there exists a cyclically reduced word u labeling a loop at 
r(l). Then both t(l)ut(l) and t(l)Mt(l) are reduced words in H, and their 
maximum common prefix is t(l). This concludes the proof. □ 

We can now state the following extension of the classical characterization of 
finite-index extensions of cyclically reduced subgroups in terms of covers. 

Proposition 1.4 Let H <fg G <fg F. Then td^) is a prefix of t{H). More- 
over, H <f\ G if and only z/ t/f (1) = tG{l) and the restriction of ip'^ is a cover 
from {cc{T (H)) , th {!)) onto {cc{T (G)) , tq (I)) ■ If that is the case, the index 
of H in G is the common cardinality of the subsets (q) (q a vertex of 

cc(r(G));. 

Proof. Let (/? = Lp'^. If u is cyclically reduced and labels a loop at a vertex p 
of cc(r(i/)), then u labels a loop at tf{p) in r(G), and that vertex is in the core 
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of T{G) by Fact 11.21 It follows that 1^(1) labels a path from the origin in r{G) 
to a vertex in cc(r(G)). In particular, tG(l) is a prefix of tff(l). 

If tG(l) is a proper prefix of t_H'(l), we have t_f/(l) = tG(l)at for some a £ A 
and t & F. Since tg{1) is in cc(r(G')), there exist a cyclically reduced word of the 
form bu, with first letter b a, which labels a loop at rG(l) in cc(r(G)). Then 
the words tG(l)(&M)"tG(l) are all reduced, and the cosets -fftG(l)(6w)"tG(l) 
are all in G. Moreover, these cosets are pairwise disjoint since H contains no 
reduced word of the form tG(l)(6w)''t^(T), d ^ 0. Thus, if H <fi G, then 
t_f/(l) = tG(l)- It follows immediately that ip maps core vertices to core vertices 
and tail vertices to tail vertices. 

Let us now assume that r(i/) and r(G) have the same tails, and let us denote 
by t(l) the word ti/(l) = tG(l). Then H' = t(T)i?t(l) and G' = t(T)Gt(l) are 
cyclically reduced, and -ff <fi G if and only if H' <f\ G' . Thus we may now 
assume that G and H are cyclically reduced. If is not a cover, there exists 
a vertex p of T{H) such that r(G) has a loop at (p{p) labeled by a cychcally 
reduced word bu {b G A) and T{H) has no 6-labeled edge out of p. Let v label 
a path from 1 to p in r{H) (and hence in T{G)). By the same reasoning as 
above, the cosets Hv{bu)^v are pairwise distinct, and contained in G. Thus, if 
H <fi G, then is a, cover. 

The converse is verified as follows: if is a cover, let wi, . . . , be reduced 
words labehng paths in T{H) from 1 to the elements 1 — pi, . . . ,pd of ip~^{l). 
If 3 £ G, then g labels a loop at 1 in r(G), and since is a cover, g labels a 
path in T{H) from 1 to pi for some i. Therefore g G Huf. thus G is the union 
of finitely many i/-cosets. □ 

Corollary 1.5 The extensions, and the finite-index subgroups of a cyclically 
reduced subgroup are cyclically reduced as well. 

2 Finite-index extensions of a subgroup H 

It follows from the characterization of finite-index extensions by covers, that if 
H <fi G, then is ontc0. Therefore H has only a finite number of finite- 
index extensions, and that number can be bounded above by the number of 
binary relations on r{H): if that graph has n vertices, then H has at most 2" 
finite-index extensions. We give a better upper bound in Section [2?5l 

Moreover, the collection of finite-index extensions of H is effectively com- 
putable. In addition, it is elementary to use these graphical representations to 
show that the join of two finite-index extensions of H is again a finite-index 
extension (see Stalling's proof of Greenberg's theorem [H]). It follows that if 
H <fg F, then H admits an effectively computable maximum finite-index ex- 
tension Hfi . The results of Section 12.41 below yield an efficient algorithm to 
compute the set of finite-index extensions of a given subgroup H , and its max- 
imum finite-index extension Hf\. 

^ The converse is not true, see [7] for a detailed study of the extensions H <fg G such that 
ifi'^ is onto. 
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Remark 2.1 We just observed that every finitely generated subgroup of F has 
a finite number of finite-index extensions. However, it usually has infinitely 
many finite-index subgroups. More precisely, every non-trivial subgroup H <fg 
F admits a finite- index subgroup of index r for each r > 1. Indeed, let A{H) = 
{r{H), 1) and let Q be the vertex set ofT{H). Define Tr be the ^-labeled graph 
with vertex set Q x {1, . . . ,r} and with the following edge set: for each edge 
{p,a,q) of r(iJ), there is an edge {{p,i), a, {q,i + 1)) for each 1 < i < r and 
an edge {{p, r), a, {q, 1)). Then Tr is an admissible graph, the map tt: (p, i) ^ p 
defines a cover from Tr to T{H), and if Hr is the subgroup represented by 
(Tr, (1, 1)), then Hr has index r in H . □ 



2.1 i-steps and finite-index extensions 

Let H < G he finitely generated subgroups of F and let 31 , . . . , g„ be reduced 
words such that G — {H, gi, . . . , gn) . Let Gq = H and let Gi = {Gi-i,gi) 
{1 < i < n). We may of course assume that gi ^ Gi-i, so Gi-i ^ Gi. 

Then A{Gi) is obtained from A{Gi-i) by, first, adding sufficiently many 
new vertices and edges to create a new path from vertex 1 to itself, labeled by 
gi] and second, by reducing the resulting graph, that is, repeatedly identifying 
vertices p and p' such that q — > p and q — > p' for some vertex q and some letter 
a € A, see for instance [TTl [121 [3 HOl ■ Depending on the length of prefixes of gi 
and gi that can be read from vertex 1 in r(G'i_i), this procedure amounts to 
one of the two following moves: 

- a reduced expansion, or re-step (we write A{Gi-i) — >re''"'*'' A{Gi)), that 
is, we add a new path labeled by a factor w oi gi, from some vertex p to some 
vertex q of r(Gi_i) in such a way that the resulting graph is admissible (needs 
no reduction); 

- or an i-step (we write A{Gi-i) — >f~'' A{Gi)), that is, we identify a pair 
of vertices (p, q) of r(G'i_i), and we reduce the resulting graph. 

Remark 2.2 Let us comment on these steps, with reference to Stallings's algo- 
rithm pjj . li H — {gi, . . . , gn) , StaUings produces A{H) by reducing (folding) a 
bouquet of n circles, labeled gi, . . . ,g„ respectively. For our purpose, we decom- 
pose this operation in n steps, adding one generator at a time and producing 
successively the A{{hi, . . . ,hi)) (1 < i < n). Each of these steps is either an 
re-step or an i-step. □ 

We refer the readers to [lOl Section 2] for a detailed analysis of these moves 
and we record the following observation. 

Lemma 2.3 Let G,H <fg F. If H <f\ G, then only i-steps are involved in the 
transformation from T{H) to T{G). 

■^This reduction operation is the iteration of Stallings's folding operation our termi- 
nology emphasizes the fact that this is a generalization of the reduction of a word (the iterated 
process of deleting factors of the form aa, a 6 A). 
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Proof. Let H — Go < Gi < . . . < Gn ~ G he as in the above discussion. Note 
that H <fi G if and only if Gi-i <fi Gi for each 1 < j < n. If A{Gi-i) — >re 
A{Gi), then the homomorphism ip^^ ^ is one-to-one, so Gi-i <fF Gi and in 
particular, Gi is not a finite-index extension of Gi-i. □ 



2.2 Which i-steps yield finite-index extensions? 

If p is a vertex of cc(r(iJ)), we let Lp{H) be the language accepted by cc{T{H)), 
seen as a finite state automaton with initial state p and all states final: that 
is, the set of (possibly non-reduced) words in A* that label a path in cc{T{H)) 
starting at vertex p. Let then Lp{H) be the set of reduced words in Lp{H) — 
which is also the set of all red(w) {u g Lp{H)), and also the set of prefixes of 
words in the subgroup represented by the pair {cc{T (H)) , p) . Let us first record 
the following elementary remark. 

Remark 2.4 If p^—>q is a path in cc(r(i/)), then Lq{H) — {red(Ma;) | x S 
Lp{H)}. □ 



We now refine the result of Lemma 12.31 

Proposition 2.5 Let H <fg F and let p^q he distinct vertices in T{H). Let G 
be the subgroup of F such that A{H) — >f~'' A{G). Then H <f\ G if and only 
if p,q are in cc{T{H)) and Lp{H) ~ Lq{H), if and only ifp,q are in cc{T{H)) 
and Lp{H) ^Lg{H). 

Proof. Let us first assume that H <f\ G and let ip — <pffj. By Proposition 11.41 
</5 is a bijection from tail(iJ) onto tail(G) and, since p(ji) = f{q), the vertices p 
and q must both be in cc(r(iJ)). 

If p, (7 e cc(r(iJ)) and Lp{H) ^ Lq{H), we consider (without loss of gener- 
ality) a word u G Lp{H) \ Lq{H), with minimum length, say u — va with a Cz A. 
By definition, there exist paths p—^p' p" and q — ^ q' , but no path q' — ^ q" 
in cc(r(iJ)). Observe that AiH) — AiG). If there is a path q'-^q" in 
T{H), then we also have A{H) — >f"=''" A{G) and since p" G cc(r(iJ)) and 
q" £ tail(r(7J)), we conclude to a contradiction by Proposition 11.41 We now 
assume that there is no path q' — ^ q" in T{H). 

We claim that there exists w E F such that aw is cyclically reduced and 
p' — >q' in cc(r(7J)). Let indeed p" — > q' be a path of minimal length in 
cc(r(7J)) (there exists one by connectedness). Since p" G cc(r(7J)), there exists 
a path p" r for some b € A, b a, and as in the proof of Remark 1 1.2 1 there 
exists a reduced word of the form bt labeling a loop at p" . Let w — red{btz): 
then we have a path p" q' . By minimality of the length of z, tb is not a 
prefix of z, so w starts with letter &, and hence aw is reduced. In fact, aw is 
cyclically reduced since there is no path q' q" . 
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Let 1 —^q' be a path in T{H). Then red{tawi) G G, and hence there exists 
m > 1 such that red{t{aw)"^t) e H. Again, since there is no path q' q" , the 
word ta is reduced. By replacing m by a sufficiently large multiple, we find that 
ta is a prefix of red (t(a-u;)™t), and hence that ta labels a path from 1 in T{H): 
this contradicts the absence of a path q' — ^ q" . 

Thus we have proved that, if H <f\ G, then p,q £ cc{T{H)) and Lp{H) = 
Lq{H). The latter condition immediately implies that Lp{H) = Lq{H). We 
now assume that p,q E cc{T{H)) and Lp{H) = Lq{H), and we show that 
H <fi G. We first establish a technical fact. 

Lemma 2.6 Let r^^^Si+i (zi £ F , < i < k) be paths in cc{T{H)), such 
that Ti, Si e {p,q} for each 1 < i < k. Then there exists a path rp '^-^ '"''t in 

cc(r(ij)). 

Proof. The proof is by induction on k, and is trivial for k = 0. If fc > 0, then 
there is a path ri ''^'^'fl^^*' f cc{T{H)). Since Lp{H) — Lq{H), there is also a 

path si for some t' G cc(r(iJ)), and therefore a path ro'^^'^^—^^'^'^t' 
as required. □ 



We want to show that G has finitely many iJ-cosets. Let u £ G: then u 
labels a loop at 1 in r(G). Let B be the automaton obtained from T{H) by 
identifying vertices p and q, but without performing any reduction. Then r(G) 
is the result of the reduction of B. In particular (say, in view of FTOl Fact 1.4]), 
u = red(u) for some word v £ A* labeling a loop at 1 in B. By definition of B, 
the word v factors as u = uq ■ ■ • Wfc, in such a way that T(H) has paths of the form 
1 si, Ti Si+i (1 < * < fc) and 1, and the vertices ri, si, • • • , r^, 

are all equal to p or q. As observed in Section [H r{H) also has paths 

red(Do) red(Di) /i / • ^ 7 \ J red(i)fc) 

1 — > Si, — > .Si+i (1 < I < fc) and — > 1. 

In particular, we have red(i;o) ~ t(l)?i;o and red(ufc) = u'fct(l) for some wq, Wk £ 
F, and there are paths 1-'^t(1)^^si and — ^ t(1) 1. Note that the 

paths r(l) si, s^+i (1 < i < /c) and — ^ r(l) are set entirely within 

cc{T{H)), since no reduced word-labeled path between vertices in cc(r(iJ)) can 
visit a vertex in tail(r(iJ)). 

By Lemma [2.6[ there exists a path t(1) ' " - — '''' t for some vertex t 

in cc(r(i?)). Let h he a shortest-length word such that t— ^t(I) in cc(r(iJ)). 
Then z = t(l) red(u;oWi • • -Vk-iWk) h t(l) labels a loop at vertex 1 in T{H), 
so red(z) G H. By construction, we have u = red(t(l)ii;oWi • • ■ Vk^iWki{l)), so 
u £ red(t(l) h t(l)). Since h was chosen to be a geodesic in cc(r(iJ)), it can 
take only finitely many values, and this completes the proof that G has finitely 
many iJ-cosets. □ 
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We note the following consequence of this proof. 

Corollary 2.7 Let H <fg F and letp,q£ cc{T{H)). Then Lp{H) = Lg{H) if 
and only if Lp{H) ~ Lq{H). 

2.3 The lattice of finite-index extensions of H 

We further refine Proposition 12.51 as follows: we consider an extension H <f\ 
G and a pair (r, s) of vertices of T{H), whose identification yields a finite- 
index extension of H. Then we show that identifying the vertices of T{G) 
corresponding to r and s, also yields a finite-index extension of G. 

Lemma 2.8 Let H <fj G <fg F and let Lp = ip'~fj. Let p be a vertex of cciT{H)). 
Then Lp{H) L^(p){G) and Lp{H) = L^^p){G). 

Proof. If p r in cc(r(iJ)), then the i^- image of this path is a path (p{p) fir), 
which is entirely contained in cc(r(G)) by Proposition ll.4l In particular, Lp{H) C 
L^(p){G). 

Conversely, suppose that (p{p) — ^ r' is a path in cc(r(G)). Since iy9 is a cover 
from cc(r(iJ)) onto cc(r(iJ)), u labels some path ^ r in cc(r{H)), and hence 
u e Lp{H). Thus Lp{H) = Z<p(p)(G). □ 

Together with Proposition l2.5l Lemma immediately implies the following 
statements. 

Corollary 2.9 Let H <fg F. 

1. Let H <fi G and let (f = (p^. Ifp,q are vertices ofT[H), A{H) — 
A{K) andA{G) — ,fP^='f^''^ A{K'), then H <f, K if and only if G <fi K' . 

2. H <fi G if and only if A{G) is obtained from A{H) by identifying some 
pairs of vertices {p,q) in cc{T{H)) such that Lp{H) ~ Lq{H), and then 
reducing the resulting graph. 

The identification of all pairs of vertices {p,q) such that Lp{H) — Lq{H) 
yields the minimum quotient of A{H) and hence the maximum finite-index 
extension H\\ of H. In addition, we find that is exactly the commensurator 
of H (the set CommF(-ff) of all elements g G F such that H D has finite 
index in both H and H^), a fact that can also be deduced from [51 Lemma 8.7]. 

Theorem 2.10 Let H <fg F and let Hf, be its maximum finite-index extension. 

1. A{Hit) is obtained from A{H) by identifying all pairs of vertices p,q of 
cc{T(H)) such that Lp{H) — Lq{H). No reduction is necessary. 

2. Hf, Comm f{H). 
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Proof. In view of Corollary 12.91 A{Hf\) is obtained from A{H) by identifying 
all pairs of vertices p,q of cc(r{H)) such that Lp{H) — Lq{H), and then by 
reducing the resulting graph B. If p — ^ r and q — ^ s (a £ A) are paths in 
cc(r(iJ)) and if Lp{H) = Lq{H), then Lr{H) = L,{H) by Remark [H Thus B 
is already reduced, which concludes the proof of the first statement. 

The fact that Comm(iJ) is a subgroup and a finite-index extension of H is 
proved, for instance, in [SJ Prop. 8.9]. Conversely, suppose that H <fi G and 
g £ G. Since conjugation by g is an automorphism of G, we have <f, G. Now 
the intersection of finite-index subgroups, again has finite index, so HnH^ <f\ G 
and hence H n Hs H and H n <fi H^. Thus g e CommF(-ff), which 
concludes the proof □ 



2.4 Computing finite-index extensions 

Recall the notion of minimization of a deterministic finite-state automaton (see 
for instance). Let B = {Q, i, E, T) be such an automaton, over the alphabet 
B, with Q the finite set of states, i € Q the initial state, E C Q x B x Q the set 
of transitions and T C Q the set of accepting states, and let L be the language 
accepted by B, that is, the set of words in B* that label a path from p to a state 
in T. Then the minimal automaton of L is obtained by identifying the pairs 
of states {p,q) such that the automata {Q,p, E,T) and {Q,q, E,T) accept the 
same language. ^ 

In our situation, the alphabet is A and Lp(H) is the language accepted by 
the automaton Bp, whose states and transitions are given by cc(r(iJ)), with 
initial state p and all states final. Therefore Corollary 12.91 and Theorem 12.101 
show that the identification of two vertices p,q £ cc(T(H)) yields a finite- index 
extension if and only if p and q are identified when minimizing ^^-(i). Moreover, 
cc(r(iJfi)) is given by the states and transitions of the minimal automaton of 

The classical Hopcroft algorithm (see [6]) minimizes an n-state automaton 
in time 0{n\ogn), so we have the following result. 

Proposition 2.11 Let H <fg F , and let n be the number of vertices o/r(iJ). 

• cc{T{Hf\)) is obtained by minimizing the automaton given by the vertices 
and edges of cc{T(H)), with all states final (the initial state does not matter 
in that situation). 

• One can compute T{Hf\) in time 0{n\ogn). 

• One can decide in time 0{n\ogn) whether identifying a given set of pairs 
of vertices ofT(H) will produce a finite-index extension of H . 

Remark 2.12 It may be that for the particular automata at hand (over a 
symmetrized alphabet, with all states final), the complexity of Hopcroft 's algo- 
rithm might be better than 0{n\ogn), even linear. It has also been observed 
that in many instances, Myhill's automata minimization algorithm exhibits a 
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better performance than Hopcroft's, in spite of a C(n^) worst-case complexity. 
Brzozowski's algorithm [2] also performs remarkably well in practice [3j. □ 

2.5 Counting finite-index extensions 

Recall that, if T is an A-labeled graph, the product F F (also called the 
fiber product, or the pull-back, of two copies of F) is the yl-labeled graph whose 
vertex set is the set of pairs {p, q) of vertices of F and whose edges are the triples 
{{p, q), a, {p', q')) such that {p, a, q) and {p' , a, q') are edges of F. This graph is 
not admissible, nor even connected in general (the vertices of the form {p,p) 
form a connected component that is isomorphic to F). Note that there is a 
u-labeled path in F F from {p, q) to {p' , q'), if and only if F has paths p-—*p' 
and q q' . 

If p, q are vertices of cc(F(i/)), we lei p ^ q if and only if Lp{H) = Lq{H). 
Proposition 2.13 Let H <fg F. 

• The relation ^ is a union of connected components of cc(F(iJ)) x ^cc(F(_H')) . 

• Let p, q be vertices of cc(T{H)). Then p ^ q if and only if the first and 
the second component projections, from the connected component of {p, q) 
in cc(T{H)) Xa cc{T{H)) to cc(T{H)) are both covers. 

Proof. The first statement follows directly from Remark 1 2. 41 which shows that 
ii p ^ q and there is a path (p, q) — >(p', g'), then p' ^ q' . 

Let us now assume that p ^ q and let us show that the first component 
projection is a cover from the connected component of (p, (?) onto cc{T{H)). 
Let (r, s) be a vertex in that connected component: then there exists u € F 
such that p-^r and q-^s. Let r-^r' (a G A) be an edge in T{H). Then 
ua G Lp{H), so ua G Lq{H), and hence (since T{H) Xa r(i?) is deterministic), 
there exists an a-labeled path s — ^ s' . Therefore there exists an a-labeled path 
{r, s) -^{r' , s'). Thus the first component projection is a cover. The proof 
concerning the second component projection is identical. 

Conversely, suppose that the first and the second component projections, 
from the connected component of {p,q) in cc(F(i?)) x^ cc(F(_ff)) to cc(F(7J)) 
are covers, and let u £ Lp{H). Then cc(F(ff)) has a path p-^r. It is 
an elementary property of covers that this path can be lifted to a path in 
cc{T{H)) Xa cc(F(i/)), of the form (p, q) — ^(r, s). The second component pro- 
jection of that path yields a path q — ^ s in cc(F(_ff)), and hence u G Lq{H). □ 

Let f{n) be the maximal number of finite- index extensions of a subgroup 
H <fg F such that F(i/) has at most n vertices. By Proposition 12.131 every 
pair (p, q) such that p ~ g is in the connected component of a pair of the form 
(l,r) for some r > 1. Moreover, this connected component has elements of 
the form (i,j) for all 1 < z < n, so the graph resulting from the identification 
of 1 and r (or from p and q) has at most n/2 vertices. Thus /(I) — 1 and 
f{n) < n/([7i/2j) for aU n > 2. It follows that f{n) < n5(i+i°g2"). 
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Proposition 2.14 Let H <fg F . If cc{T{H)) has n vertices, then H has at 
most n^'^-'^+'°S2 ") finite-index extensions. 

Example 2.15 By means of lower bound, we consider the following example. 
Let ei, . . . , Gfe be the canonical basis of the vector space Z|, let ip be the mor- 
phism from the free group F over A = {ai, . . . ,aj,} into the additive group 
1j\ , mapping to , and let H = ker (p. Then H is normal and finite-index, 
so all its extensions have finite index and they are in bijection with the set of 
quotients of Zj, hence with the set of subgroups of Zg, or equivalently with the 
set of subspaces of Zj. 

Let £d,k be the number of linearly independent d-tuples in Z2 {d> 1). Then 
£i^k = 2^^ — 1. If d > 2, a d-tuple (xi, . . . , Xd) is linearly independent if and only 
if (a; 1, ... , Xd~i) is linearly independent and Xd does not belong to the subspace 
generated by xi, . . . ,Xd-i, so that £d,k = ^(i-i,fc(2*' — 2'^~^). Now the set of 
cardinality d linearly independent subsets has md,k = ^d,k/d\ elements, and the 
number of dimension d subspaces of Z| is 

^ nidM ^ ^ ^ (2fc-l)(2^-2)(2fc-4)---(2fe-2'^-i) 
md.d IdA (2'i-l)(2''-2)(2'i-4)...(2'i-2'i-i)- 

Finally, the number of subspaces of Zj is equal to X]d=o ^d,k^ with so,fe = 1. 
We observe that for each < i < d < /c, |rE|- > 2'="'^, so that Sd,fe > 2'-^-'^'^'^. 

By considering d = [^\ , we find that ^ Sd^k > 2*^ 

Finally, we note that r{H) is the Cayley graph of Z2 with respect to the basis 
ei, . . . , (a graph known as the dimension k hypercube), so that T(H) has n = 
2^ vertices. As a result, Fl has more than n3'°S2" finite-index extensions. □ 

2.6 The lattice of finite-index subgroups of G 

Let us call W-maximal a subgroup G <fg F which has no proper finite- index 
extension, that is (in view of Theorem l2.10p . such that G — Commi?(G'). If G 
is fi-maximal, let FIS(G) be the set of all finite-index subgroups of G, that is, 
the set of subgroups H <fg F such that i/fi = G. Note that distinct fi-maximal 
subgroups yield disjoint lattices of finite-index subgroups. 

Remark 2.16 Let G <fg F be non-trivial and fi-maximal. Then FIS(G) forms a 
convex sublattice of the lattice of subgroups of F, with greatest element G. This 
sublattice is always infinite (see Remark l2.ip and without a least element. □ 

Lemma 12.81 provides us with an invariant for every sublattice of the form 
FIS(G) (with G fi-maximal). 

Proposition 2.17 Let H,K <fg F. Then Hf, = Kf, if and only if tnil) = 
t/f (1) and Lt-(i){H) — Lt(i){K), if and only if t/f (1) = ^^(1) o,nd L^(i)(iJ) = 

Lril){K). 
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Proof. If Hi] ~ Kf], then H,K <f, Hf-,, and Proposition 11.41 and Lemma [2.81 
show directly that tff(l) = tH,,(l) = tR^, Lr(i){H) Lr{i){Hf\) = Lr{i){K) 
and = L^(i)(i?fi) = 

We now prove the converse. More precisely, we show that if H and K are 
fi-maximal, t//(l) = tif(l) and = then H — K. First we 

note that = (cc(r(i?)), t(1)) and in particular, i7tff(i) jg cyclically 

reduced and fi-maximal. Thus, it sufRces to prove the expected result (namely, 
that H = K) under the hypothesis that H and K are cyclically reduced. 

By Remark 12.41 the set of all Lp{H) {p G T{H)) coincides with the set 
{{red(ux) \ x e Li{H)} \ u E Li{H)}. In addition, since H is fi-maximal, if 
u,v £ Li{H) with \^^p and l^^q and {red('ita;) | x G Li{H)} = {red(ux) | 
X S Li(7J)}, then p — q. It also follows from the same fact that, again if H is 
fi-maximal, there is an edge {p,a,q) in T{H) if and only if Lq{H) = {red(ax) | 
X £ Lp{H)}. Thus, the cyclically reduced fi-maximal subgroup H is entirely 
determined by the set Li{H). This concludes the proof. □ 

The pairs {t, L) that are equal to (t(l), _Lt.(i)(G')) for some subgroup G <fg F 
are characterized as follows. Recall that an i-automaton |5] is a deterministic 
automaton (Q,i, E,T) over the alphabet A such that, {p,a,q) G i? if and only 
if {q,a,p) G E for all vertices p, g and a £ A. The automata Bp discussed in 
Section 12.41 are i-automata. 

Proposition 2.18 Let t € F and let L C A* be a rational language. The 
following conditions are equivalent. 

(1) There exists a subgroup H <fg F such that t — t(l) and L — L^(-x)(i?). 

(2) There exists a Y\-maximal subgroup H <fg F such that t = t{l) and L = 

(3) L is accepted by an i-automaton with all states accepting and such that, 
for each state p, there exist transitions {p, a, q) and [p, b, r) for at least 
two distinct letters a,b £ A. In addition, if t is not the empty word, then 
t — t'a for a letter a £ A such that a <^ L. 

(3') The minimal automaton of L is an i-automaton with all states accepting 
and such that, for each state p, there exist transitions {p, a, q) and {p, b, r) 
for at least two distinct letters a,b £ A. In addition, if t is not the empty 
word, then t — t'a for a letter a £ A such that a ^ L. 

(4) t and L satisfy the following conditions: 

(4-1) L is closed under taking prefixes; 

(4-2) if u,v £ L, then uuv £ L; 

(4-3) if uvvw £ L, then uw £ L; 

(4-4) if ua £ L with a £ A, then uab £ L for some b £ A such that b ^ a. 
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(4-5) if t is not the empty word, then t = t'a for a letter a € A such that 
a ^ L. 

Proof. Conditions (1) and (2) are equivalent by Proposition l2.17l 

Proposition 12.111 shows that (1) imphes (3'), which in turn implies (3). Let 
us now assume that (3) holds and let T be the A-labeled graph induced by the 
states and transitions of the minimal automaton of L. The extra condition given 
shows that every vertex of F is visited by a loop labeled by a cyclically reduced 
word. It follows that, if G is the subgroup whose graphical representation is 
A{G) — (r, go), with qq the initial state, then G is cyclically reduced, L = 
Lqg{G). The condition on the word t shows that Condition (1) holds with 
H = G*. 

Condition (3) easily implies Condition (4). Let us now assume that Con- 
dition (4) holds. By [9, Theorem 4.1], Properties (4.2) and (4.3) show that L 
is accepted by an i-automaton. Property (4.1) shows that all states of that 
automaton are final, and Property (4.4) shows that, for each state p, there exist 
transitions (p, a, q) and (p, 5, r) for at least two distinct letters a,b G A. Thus 
Condition (4) implies Condition (3), which concludes the proof □ 

3 Malnormal closure 

A subgroup of F is malnormal if H H = 1 for each g ^ H. Malnormality 
was proved decidable in [1], and a simple decision algorithm was given in [5], 
based on the following characterization [H |4] . 

Proposition 3.1 Let H <fg F. Then H is malnormal if and only if every 
connected component ofT{H) XaT{H), except for the diagonal complement, is 
a tree, if and only if, for every p ^ q (z cc(T{H)), L,p{H) D Lq{H) is finite. 

This yields directly an 0{n^ ^ogn) decision algorithm, where n is the number 
of vertices of cc{T{H)). It also yields the following corollary. 

Corollary 3.2 Let H <fg F. If H is malnormal, then H is f'\-maximal, that is, 
H has no proper finite-index extension. 

Proof. By Corollarv l2.9i if H is not fi-maximal, then there exist vertices p ^ q 
in cc{T[H)) such that Lp[H) = Lq{H), and hence such that Lp{H) fl Lq{H) = 
Lp{H) is infinite. In particular, H is not malnormal. □ 

It is shown in [7, Prop. 4.5] that for every finitely generated subgroup 
H <fg F, there exists a least malnormal extension i?mah called the malnormal 
closure of H, that H^ai is finitely generated and effectively computable, and 
that the rank of iJmai is less than or equal to the rank of 7J [71 Corol. 4.14]. In 
fact, [7] shows that T{Hma\) is obtained from r{H) by a series of i-steps. The 
algorithm computing H^ai then consists in computing all the quotients of ^{H) 
and verifying, for each of them, whether it represents a malnormal subgroup. 
We now give a much better, polynomial-time algorithm. 
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Theorem 3.3 Let H <fg i^„. The malnormal closure of H is computed by 
repeatedly applying the following construction: in T{H), identify all pairs {p,q) 
of distinct vertices in cc{T{H)), such that Lp{H) n Lq{H) is infinite and reduce 
the resulting graph. 

Proof. Let Hq = H and let be the subgroup of F such that T{Hi+i) is 

obtained from T{Hi) by, first identifying aU pairs p,q of vertices of cc(r(_ffi)) 
such that Lp(Hi) D Lq{Hi) is infinite, and then reducing the resufiing graph. 
Since T{Hi+i) has less vertices than r{Hi), this defines a finite sequence of 
subgroups 

H = Ho<Hi<...<Hk, 

where fc > and Hj. is malnormal. We want to show that Hk is the least 
malnormal extension of H . 

Let K be any malnormal subgroup of F such that H < K. By Takahasi's 
theorem (see for instance [7 ), there exists a subgroup G such that H < G < K 
such that G is obtained from H by a sequence of i-steps, and K is obtained 
from G by a sequence of re-steps (this fact can also be deduced from ^TU\ Prop. 
2.6]). 

Let p,q ^ cc(r(i7)) such that Lp{H) n Lq{H) is infinite. It is elementary to 
verify that Lp{H) C L^G(p)(G) C L^g(^G(p))(i4:). In particular, <^g{Vh{p)) ^ 
fciv'ni'i)) is infinite. Since K is malnormal, it follows from Proposition 13.11 
that ((/3^(p)) = ((/?^((7)). But is one-to-one by definition, so we have 
fnip) — 'Pni^)- I* follows that G is obtained from Hi by a sequence of i-steps, 
and in particular. Hi < G < K . Iterating this reasoning, we find that Hk < K, 
which concludes the proof □ 

Corollary 3.4 Let H < F. Lf T{H) has n vertices, then one can compute 
r(i?mai) in time 0{n^ \ogn) . 

Proof. According to the algorithm given in Proposition 13.31 we first need to 
compute the connected components of cc(r(if)) Xa cc(r{H)) - done in time 
0{n^ logn) -, identifying which are trees and which are not - done in time O(n^) 
-, identifying the vertices of cc(r(i?)) involved in a non-diagonal connected 
component and reducing the resulting graph to obtain r(_ffi) - which is done 
in time 0{n\ogn). Thus r(-ffi) is computed from r{Ho) in time 0{n^ logn). 

This part of the algorithm is iterated k times, to compute r{Hk) — r(iJrnai), 
and we have k < n since the number of vertices of the T(Hi) forms a properly 
decreasing sequence. This concludes the proof. □ 
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